Search for: All records

We study the problem of synthesizing a controller that maximizes the entropy of a partially observable Markov decision process (POMDP) subject to a constraint on the expected total reward. Such a controller minimizes the predictability of an agent’s trajectories to an outside observer while guaranteeing the completion of a task expressed by a reward function. Focusing on finite-state controllers (FSCs) with deterministic memory transitions, we show that the maximum entropy of a POMDP is lower bounded by the maximum entropy of the parameteric Markov chain (pMC) induced by such FSCs. This relationship allows us to recast the entropy maximization problem as a so-called parameter synthesis problem for the induced pMC. We then present an algorithm to synthesize an FSC that locally maximizes the entropy of a POMDP over FSCs with the same number of memory states. In a numerical example, we highlight the benefit of using an entropy-maximizing FSC compared with an FSC that simply finds a feasible policy for accomplishing a task. 

The problem of synthesizing an optimal sensor selection policy is pertinent to a wide variety of engineering applications, ranging from event detection to autonomous navigation. In this paper, we consider such a synthesis problem in the context of linear-Gaussian systems. Particularly, we for- mulate the optimal sensor selection problem in terms of a value iteration over the continuous space of covariance matrices. To obtain a computationally tractable solution, we subsequently formulate an approximate sensor selection problem, which is solvable through a point-based value iteration over a finite “mesh” of covariance matrices with a user-defined bounded trace. In addition, we provide theoretical guarantees bounding the suboptimality of the sensor selection policies synthesized through this approximate value iteration. Finally, we analyze the efficacy of our proposed method through a numerical example comparing our method to known results.